Methods and systems for improving low-resolution video

ABSTRACT

Systems and methods are provided for improving the visual quality of low-resolution video displayed on large-screen displays. A video format converter may be used to process a low-resolution video signal from a media providing device before the video is displayed. The video format converter may detect the true resolution of the video and deinterlace the video signal accordingly. For low-resolution videos that are also low in quality, the video format converter may reduce compression artifacts and apply techniques to enhance the appearance of the video.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit under 35 U.S.C. §119(e) of U.S.Provisional Application No. 60/878,967, filed Jan. 5, 2007, which isincorporated herein by reference in its entirety.

BACKGROUND OF THE INVENTION

With the increasing use of portable media players, such as MP3 playerswith video capability, videos are now commonly created for use withthese devices. For example, online stores are providing movies,television episodes, and other video content for download to portablemedia devices. The video content provided for these devices may havevideo characteristics that are suitable for display on the small displayscreens of these portable devices.

Another form of video content that is also becoming popular is onlinevideo. In particular, many television shows and movies are readilyavailable for download on demand. Furthermore, many users shareuser-generated videos to the public through various video-sharingwebsites. Conventionally, these Internet-provided videos are displayedon personal computer (PC) or laptop monitors, where screen sizes arerelatively small. Therefore, online video, as well as videos generatedfor portable media players, typically have lower resolution.

In addition to having lower resolution, user-generated videos on theInternet are often created by amateurs who are unfamiliar with or unableto use professional techniques in video-generation. For example,user-generated videos may be filmed on hand-held video cameras. Thus,due to the shaking camera, the frame-by-frame video may change even forunchanging backgrounds. For a given data rate, this causes adisproportionate amount of compressed video data to be used onbackgrounds and other still images. For this and other reasons,user-generated content often suffers not only from low resolution, butalso from artifacts. Artifacts are referred to herein as visuallydispleasing portions in a display that are caused by video compression.Common artifacts include blocking artifacts and mosquito noise. Blockingartifacts refer to the blocky appearance of a low resolution video thatis typically seen on areas of less detail in the image. Mosquito noiseis a ringing effect, caused by truncating high-frequency luminanceand/or chrominance coefficients, typically seen around sharp edges inthe video.

With improvements in technology, and in particular networking, Internetcontent and portable media player content may be displayed on otherdevices, such as on television screens. However, these other devices mayhave larger screens, and therefore higher resolution, than that of thevideo content provided by the Internet or portable media players.Therefore, when videos with lower resolution, and in some cases withcompression artifacts, are blown up to a larger size, the picturequality may become unacceptably poor, creating an unpleasant viewingexperience for a user.

Furthermore, processing techniques performed by large-screen displaydevices when preparing a video signal for display on the device mayworsen the presentation of low-resolution video. One such processingtechnique performed by display devices is deinterlacing, a process thatchanges the way that pixels are drawn on a screen. Videos are displayedby a display device by drawing successive images at fast enough rate(e.g., 50 frames per second). Typically, a display presents these imagespixel by pixel using either a progressive or interlaced scan. Aprogressive scan draws out each pixel in an image from the top of thescreen to the bottom. Thus, after each scan, a progressive displaydisplays an entire frame. An interlaced scan, on the other hand, drawsout the odd pixel lines in an image. Then, at the next time instant, theeven pixel lines are drawn out. An interlaced scan, therefore, creates avideo by alternating between displaying the odd lines and displaying theeven lines of successive images. These half-resolution images are calledfields.

Currently, many display devices (e.g., some digital televisions, liquidcrystal displays (LCDs), etc.) are progressive displays. However, videotransmission standards, such as television broadcast standards, commonlyuse interlacing. Therefore, these display devices often includedeinterlacing circuitry for converting interlaced videos to progressivevideos. There are several different deinterlacing techniques employed bydigital display devices. These techniques attempt to display aninterlaced video with the highest possible visual quality. Thus, toeffectively display television broadcasts and other interlaced videos,the deinterlacing circuitry in televisions and other display devices arebecoming increasingly complex and sophisticated.

In general, because of these deinterlacing and other new, sophisticatedtechniques for effectively processing higher-resolution video signals,viewers have come to expect vivid and high-quality images on theirtelevision sets. In particular, these techniques are being incorporatedinto regularly available television sets for displaying large imageswith higher brightness, contrast, and resolution. However, these complexprocessing techniques may not be effective when performed onlow-resolution, and possibly low-quality, videos, such as content fromthe Internet or portable media players. In fact, these techniques mayworsen the presentation of low-resolution videos. Therefore, there iscurrently no effective way to present both higher-resolution videocontent, such as television broadcasts, and low-resolution videocontent, such as Internet content, on a large-screen device. Thus, itwould be desirable to improve the visual quality of low-resolution videoon large-screen displays.

SUMMARY OF THE INVENTION

Accordingly, systems and methods are provided for improving the visualappearance of low-resolution video displayed on high-resolution,large-screen devices.

In accordance with one aspect of the invention, a video format converteris used to process a low-resolution video before it is processed by alarge-screen device. Often times, low-resolution video is transmittedusing a video transmission standard for higher resolution videos, so thelow-resolution video appears to possess a higher resolution that itactually has. Thus, to determine whether to deinterlace and/or processthe video signal using a low-resolution technique, the video formatconverter may detect the true resolution of the video content. In someembodiments, if the video converter detects that the video islow-resolution and interlaced, the video converter's deinterlacer maydeinterlace the received video signal using a technique determined basedon the true resolution. In other embodiments, the video converter'sdeinterlacer may convert the received video signal to its trueresolution and deinterlace the converted video signal. Either way, thevideo converter produces a progressive video signal (e.g., in HDMI orDVI format, etc.). Therefore, the deinterlacing circuitry of thelarge-screen device, which may be unsuitable for low-resolution video,is avoided.

In accordance with another aspect of the present invention, the videoformat converter may take additional processing steps to improve theappearance of low-resolution videos that are also low in quality. Insome embodiments, to determine whether a video signal needs additionalprocessing, the video converter may look for a low-quality signature inthe video signal. If the video converter determines that a video signalis low-quality, it first reduces the artifacts in the video signalusing, among other techniques, MPEG noise reduction for reducingmosquito noise and blocking artifacts. After noise reduction, the videosignal is left with very little information. The reduced video signal isthen enhanced to improve the appearance of the noise-reduced video. Insome embodiments, enhancing the video signal involves increasing thecontrast of the picture. In some embodiments, enhancing the video signalinvolves adding film grain to create the illusion of texture and tocover up deficiencies in the picture. These and other techniques forenhancing the video signal improve the visual perception of a viewerwhen there is little detail in the picture.

BRIEF DESCRIPTION OF THE FIGURES

The above and other objects and advantages of the invention will beapparent upon consideration of the following detailed description, takenin conjunction with the accompanying drawings, in which like referencecharacters refer to like parts throughout, and in which:

FIG. 1 shows a video providing device coupled to a display device;

FIG. 2 illustrates a number of video providing devices that may connectto a display device;

FIG. 3 illustrates the difference between 2-dimensional (2D) and3-dimensional (3D) deinterlacing;

FIG. 4 illustrates a system using a dock or other hardware forprocessing a video signal;

FIGS. 5-6 are illustrative flow diagrams for processing a low-resolutionvideo signal;

FIGS. 7-8 are illustrative flow diagrams for improving the visualquality of a low-resolution video.

FIG. 9A is a block diagram of an exemplary hard disk drive that canemploy the disclosed technology;

FIG. 9B is a block diagram of an exemplary digital versatile disc thatcan employ the disclosed technology;

FIG. 9C is a block diagram of an exemplary high definition televisionthat can employ the disclosed technology;

FIG. 9D is a block diagram of an exemplary vehicle that can employ thedisclosed technology;

FIG. 9E is a block diagram of an exemplary cell phone that can employthe disclosed technology;

FIG. 9F is a block diagram of an exemplary set top box that can employthe disclosed technology; and

FIG. 9G is a block diagram of an exemplary media player that can employthe disclosed technology.

DETAILED DESCRIPTION

FIG. 1 shows illustrative system 100 for providing video content to adisplay device. Display device 104 in system 100 may be a television orany other device that can display video. Video content 106 may beprovided by video providing device 102. Video providing device 102 maybe a portable media player, a DVD player, a set top box, or any othersuitable device that may provide video. Video content 106 may be storedin memory within video providing device 102, or video providing device102 may obtain video content 106 directly from an external source (e.g.,the Internet, a DVD, etc.). Video content 106 may have any suitableresolution (e.g., 320×240, 160×120, 640×480, etc.), may use any suitableencoding (e.g., uncompressed, H.264, MPEG, etc.), and may have any othersuitable video characteristics.

Video providing device 102 includes processing circuitry 108. Processingcircuitry 108 converts media content 106 into a video signal suitablefor transmission to display device 104. The video signal may be of anysuitable format. For example, the video signal may be in compositevideo, S-Video, or component video (e.g., YPbPr, RGB, etc.) format. Thevideo signal may utilize a digital format such as a high-definitionmultimedia interface (HDMI) or digital video interface (DVI) format.Processing circuitry 108 may include an encoder to map video content 106to a video signal of a given video transmission standard. Processingcircuitry 108 may include, for example, National Television SystemsCommittee (NTSC), phase alternating line (PAL), or SECAM encoders. Forsimplicity, when appropriate, “video content” and “video signal” mayhereinafter be used interchangeably. For example, a “low-resolutionvideo signal” refers to a video signal corresponding to low-resolutionvideo content.

The video signals from processing circuitry 108 may be transmitted todisplay device 104 using link 110. Link 110 may be one or more cables orother wired connections. Link 110 may also be a wireless connection. Thetransmitted video signal is received from link 110 by display device104. Display device 104 may include processing circuitry 112 and displayscreen 114. Display screen 114 displays video content 106 to a user.Display screen 114 may have any suitable resolution (e.g., 640×480,1280×720, 1920×1080, etc.). Processing circuitry 112 processes the videosignal received from link 110 and prepares the video for display ondisplay screen 114. Processing circuitry 112 may process the videosignal based on the resolution and other characteristics of displayscreen 114, and may attempt to improve the visual quality of the video.

For progressive displays, or displays that show full frames at each timeinstant, processing circuitry 112 includes circuitry to deinterlaceinterlaced video. An interlaced video is one that is composed of twotypes of fields: an odd field consisting of the odd lines of pixels inan image and an even field consisting of the even lines in a image. Tocreate an interlaced video, the two types of fields are displayed in analternating fashion at each time interval (e.g., every 16.67milliseconds for NTSC, every 20 milliseconds for PAL, etc.). Thus, oddfields are displayed at every other time interval, and even fields aredisplayed in the remaining time intervals. Since only half the pixels ina display are utilized at any given time instant, an interlaced videohas at most half the resolution of the display. A progressive video, onthe other hand, may utilize up to the full resolution of the display.

Converting an interlaced video to a progressive video involves a processreferred to as deinterlacing. Deinterlacing involves determining oddpixel lines in an even field and/or determining even pixel lines in anodd field using incomplete information. More properties of displaydevice 104 will be discussed below in connection with FIG. 2.

System 200 in FIG. 2 is an illustrative system that includes multipletypes of video providing devices. The video providing devices in system200 can include portable media player 208 (e.g., video MP3 player,video-ready mobile phone, etc.), DVD player 210, set top box 206, videocassette recorder (VCR) 212, and computer/laptop device 204. It shouldbe understood that any other type of video providing device may beincluded in system 200, and therefore system 200 is not limited to thevideo providing devices shown in FIG. 2. For example, system 200 mayinclude video providing devices other than DVD player 210 that supportremovable digital disks (e.g., HD-DVD and Blu-ray disks). Thus, althoughreference is made to DVDs in various embodiments throughout thisdisclosure, it should be understood that these embodiments may beapplied to HD-DVD and Blu-ray.

Each video providing device in system 200 may support one or more of theformats and standards described above in connection with FIG. 1, or anyother suitable format or standard. For instance, portable media player208 may provide video signals in composite and S-Video formats via link214. Link 214 may include any number of wired (e.g., cables, etc.) orwireless links. While portable media player 208 is shown in FIGS. 2 and4 to provide video signals in composite and S-Video formats, this ismerely one illustrative example. In some embodiments, portable mediaplayer 208 may support other suitable output formats instead of or inaddition to composite and S-Video, such as any of the formats discussedabove in connection with FIG. 1. For example, portable media player 208may support component video that is either interlaced or progressive,and/or may support one or more digital video interfaces, such as HDMI orDVI.

As shown in FIG. 2, display device 104 may have one or moresockets/interfaces 202 (not pictured in FIG. 1) to receive videosignals, where each interface may support one or more of the videoformats described above in connection with FIG. 1. For example, displaydevice 104 may have separate input sockets for composite video, S-Video,component video, HDMI, and DVI. Therefore, any video providing device insystem 200 may provide video content via one or more of sockets 202. Iflink 214 is wireless, sockets/interfaces 202 may be a network interface.In system 200, portable media player 208 is shown to be coupled todisplay device 104 through the composite video and S-Video sockets, butany of the other video providing devices may be coupled to displaydevice 104 using one or more of these or other sockets. Thus, multipletypes of video providing devices may provide video content to displaydevice 104 using the same interface.

Processing circuitry 112 (FIG. 2) receives a video signal from one ofsockets 202. Processing circuitry 112 processes the received videosignal to display video content on display screen 114. For example, foran interlaced video signal, processing circuitry 112 may use3-dimensional (3D) deinterlacing to convert a received video signal to aprogressive format. 3D deinterlacing may obtain full frames fromhalf-resolution fields by spatial interpolation (e.g., pixelreplication, averaging neighboring pixels, etc.) and/or by temporalcombining (e.g., combining an odd and even field, etc.). Thus, 3Ddeinterlacers may utilize all relevant information (e.g., spatial andtemporal) in order to deinterlace a video signal. This processingtechnique is suitable for videos that are already somewhathigh-resolution and high-quality, such as videos meant for display on atelevision (e.g., a television broadcast received by set top box 206, acommercial movie from DVD player 210, etc.).

However, the characteristics of video content often vary depending onthe type of video providing device. Although many devices providehigh-resolution (e.g., 720×480) and high-quality(professionally-generated) videos, many video providing devices mayprovide lower-resolution (e.g., 320×240, 160×120, etc.) and/orlower-quality (e.g., amateur-generated) videos. For example, portablemedia player 208, due to its small screen size, may provide videos thathave low resolution. Computer/laptop device 204 and set top box 206 mayprovide Internet content, which are often low-resolution and/oruser-generated. Furthermore, any of the video providing devices shown insystem 200 (FIG. 2) may provide video content that is highly compressed,and therefore low in quality. In general, due to the variety of videoproviding devices and video compression/encoding algorithms, theresolution and quality of videos provided to display device 104 may varyconsiderably. Processing circuitry 112, however, is typically not awareof the origins of a video signal, and may therefore process alow-resolution video signal in substantially the same way as ahigh-resolution signal. These processing techniques, such as theabove-described 3D deinterlacing technique, may not be as effective forlow-resolution video signals. In fact, in some instances, applyingprocessing techniques intended for high-resolution video may actuallyreduce the visual quality of a low-resolution video signal.

FIG. 3 illustrates one adverse affect that may result from using the 3Ddeinterlacing circuitry of display device 104 for a low-resolution videoprovided by, for example, portable media player 208. Portable mediaplayer 208 may store and display videos that have a low resolution suchas 320×240. One frame (or two combined fields) of a 320×240 still videois represented by image/frame 302, where each square represents a pixel.That is, image 302 represents a series of unchanging images in a video,such that the resulting video is unmoving for a period of time. In theevent that a user wants to watch a video stored in portable media player208 on another display device (e.g., display device 104), the videocontent may be transmitted as a video signal through link 214. In someembodiments, the transmitted video signal is of a standard format thatrequires the video signal to be transmitted at a higher resolution(e.g., 640×480). Thus, portable media player 208 may simply increase(e.g., double) the number of lines and the number of pixels in eachline. Image 304 represents the odd field of image 302 after conversionto 640×480. The even field is the same as the odd field. Clearly, image304 does not have 640×480 resolution even though it is being transmittedas such. Processing circuitry 112 of display device 104, unaware of theorigins of the video signal, may blindly apply 3D deinterlacing at 306as if the video were a true 640×480-resolution video. Processingcircuitry 112, therefore, notices that the video is unchanging insuccessive time instances, and combines the odd and even fields. For anormal 640×480 video, the combination would produce a full-resolutionframe with complete information. 3D deinterlacing for the 320×240 videotransmitted as 640×480, however, produces image 308. Note that image 308is not an improvement upon the original image 302, and displayed on alarger scale, is visibly jagged and blocky.

For a low resolution video transmitted with a higher resolutionstandard, 2D deinterlacing may produce results that have better visualquality. 2D deinterlacing does not utilize temporal information tointerpolate, and instead interpolates unknown pixels based onsurrounding, known pixels. This form of deinterlacing may be moreappropriate for the 320×240 video transmitted at 640×480, since noinformation is gained from combining the odd and even field. In someembodiments, 2D deinterlacing may involve vector interpolation, whereedges in each image (e.g., the outline of objects, etc.) are determined,the angles of the edges are calculated, and unknown pixels areinterpolated from neighboring pixels lying along the calculated edgeangles. Performing 2D deinterlacing with vector interpolation at 310 mayresult in image 312. Image 312 may be an improvement over image 308,because the jagged and blocky edges have been visibly smoothened. Vectorinterpolation and its functionalities are described in greater detail inSahu et al. U.S. patent application Ser. No. 11/294,709, which is herebyincorporated by reference herein in its entirety.

Therefore, to improve the presentation of low-resolution and potentiallylow-quality videos, the video signals corresponding to low-resolutionvideos may be processed prior to reception by processing circuitry 112(FIG. 1). Instead of directly feeding a video signal from a videoproviding device to display device 104, as shown in FIGS. 1 and 2, thevideo signal may first be processed by a video format converter. In someembodiments, the video format converter may be embedded in a deviceexternal to both the video providing device and the display device. FIG.4 shows illustrative system 400 that utilizes such an external device,e.g., dock 408. Low-resolution video content may be provided by portablemedia player 208, which is one type of video providing device 102. Thelow-resolution video content may be stored in storage 402 within theportable media player. Processing circuitry 404 may convert the storedvideo content into video signals using any of the techniques describedin connection with processing circuitry 108 in FIG. 1. The video signalsmay be transmitted to dock 408 through link 214. Converter 410 mayconvert video signals from link 214 to video signals 412 that, whendisplayed on display device 104, will be more visually pleasing. In someembodiments, video format converter 410 may convert composite video orS-Video outputs from link 214 into a progressive format at 412, such asHDMI. Therefore, video format converter 410 may perform deinterlacingusing a technique that is appropriate for the particular resolution andquality of a video signal. For the example discussed above in connectionwith FIG. 3, video format converter 410 may deinterlace the video signalusing 2D rather than 3D deinterlacing. Therefore, because the video istransmitted to display device 104 using a progressive format, thepotentially unsuitable deinterlacing circuitry of display 104 may beavoided.

Dock 408 may additionally contain circuitry or functionalities otherthan video format converter 410. If link 214 is wireless, dock 408 maycontain a network interface to receive and process a wireless videosignal. In some embodiments, a received video signal may be in acompressed format (e.g., H.264, MPEG4, VC-1, MPEG2, etc.). Dock 408 maytherefore include circuitry that decompresses the compressed videosignal and provides the decompressed video signal to video converter410. In some embodiments, dock 408 may support receiving multiple videoformats using one link (e.g., one cable, a wireless link). Then, dock408 may additionally include a multi-format decoder (not pictured). Themulti-format decoder may pre-process a received video signal based onthe format of the video. For example, decoding a video signal mayinvolve decompressing a compressed video signal according to the type ofcompression used, as described above.

In some embodiments, dock 408 in FIG. 4 is designed for a specific typeof portable media player (e.g., a video MP3 player). For example, dock408 may include an interface for coupling the portable media player tothe device. This interface may be shaped to only accept input from onebrand or one type of portable media player. Alternatively, the interfacemay include a proprietary connector. In other embodiments, the interfacefor dock 408 may support a set of portable media players, a set of DVDplayers, a set of computer/laptop devices, or a set of any other type ofvideo providing device. In still other embodiments, dock 408 may supporta set of video providing devices (e.g., all video providing devices thatprovide NTSC, etc.). Thus, the invention described herein is not limitedto a portable media player, but may be applied to any device thatprovides low-resolution video. It should therefore be understood thatdock 408 can include any type of interface (e.g., physical connector,network, proprietary, etc.) for coupling an electronic device to thedock.

Furthermore, video format converter 410 (FIG. 4) and any other circuitrydescribed in connection with dock 408 need not be embedded in a devicethat is external to a video providing device and a display device. Insome embodiments, video format converter 410 may be part of processingcircuitry 112 (FIG. 1) in display device 104. For example, video formatconverter 410 may be embedded within a television, and may selectivelyprocess received video signals based on the resolution and/or quality ofthe video. Alternatively, video format converter 410 may process allreceived video signals regardless of the resolution/quality. In otherembodiments, video format converter 410 may be part of the processingcircuitry in a video providing device. For example, video formatconverter 410 may be embedded in a computer/laptop device, and mayselectively process video signals prior to transmission based on theresolution and/or quality of the video. Alternatively, video formatconverter 410 may process all video signals prior to transmissionregardless of the resolution/quality. The video format converterembedded in a device may be enabled or disabled by a user (e.g., bypushing a button on a television remote, by selecting a setting on acomputer, etc.).

Flow diagram 500 in FIG. 5 shows illustrative steps that video formatconverter 410 (FIG. 4) may take to deinterlace and process a videosignal. Video converter 410 receives a video signal at step 502. Thevideo signal may be of any suitable format. At step 504, video converter410 may detect the resolution of the video signal. In some embodiments,video converter 410 may only take video input from a particular type orbrand of a video providing device. For example, dock 408 (FIG. 4) mayinclude an interface that is shaped to fit one type of device, or videoconverter 410 may be embedded in a particular video providing device. Inthese embodiments, resolution detection at step 504 may be simpler,since the resolution may be hard-coded or hard-wired based on the knownencoding of the video providing device. Further proprietary or othermore advanced connections may include embedded resolution information.In some embodiments, and for certain types of products, the resolutioninformation could also be provided in response to a user input, such asin response to the user explicitly selecting content of particularresolutions. For example, when downloading online content, one can oftenexplicitly choose which resolution to download the online content. Forthe example discussed in connection with FIG. 3, video format converter410 may be hard-wired or hard-coded to expect the resolution of everytransmitted video to be 320×240, or equivalently, to expect the actualresolution to be half of the transmitted resolution.

In other embodiments of system 400 in FIG. 4, video format converter 410may be used for a plurality of video providing devices. Therefore, theresolution or scaling factor of the video signal received at step 502 inflow diagram 500 (FIG. 5) may vary. In this case, resolution detectionof the video signal at step 504 is more complex. The video convertermay, for example, have a front-end circuit that detects pixelreplication in the video signal. The detector may compare verticallyand/or horizontally neighboring pixels, or the detector may compare anumber of surrounding pixels. The pixel radius, or a similar metric,that is used for a given comparison may be programmable. Furthermore, athreshold for the percentage of pixels that should match in order toform a confident determination of the resolution may also beprogrammable. For the example discussed above in connection with FIG. 3,video converter 410 may detect that the received 640×480 video signalactually has a resolution of 320×240. If a cell phone with a resolutionof 160×120 is coupled to video converter 410 instead, video converter410 may detect the true resolution or detect that the true resolution isone-fourth of the transmitted resolution (assuming again that thetransmitted resolution is 640×480).

At step 506 in FIG. 5, the received video signal may be deinterlaced.The video signal may be deinterlaced in its transmitted resolutionregardless of any discrepancies between the transmitted and the actualresolution. However, the technique used to deinterlace the video signalmay be selected based on the detected resolution. For the example asdiscussed in connection with FIG. 3, the transmitted, 640×480-resolutionvideo may be directly deinterlaced by video converter 410, and 2Ddeinterlacing may be used because of the detected, 320×240 resolution.In other scenarios, deinterlacing may involve other forms of 2Ddeinterlacing, some form of 3D deinterlacing, or any other deinterlacingtechnique. An advanced form of 3D deinterlacing is discussed in greaterdetail in U.S. patent application Ser. No. 11/932,686, which is herebyincorporated by reference herein in its entirety.

Deinterlacing at step 506 in FIG. 5 may additionally involve scaling thevideo to an image size suitable for display on a television or otherlarge-screen display device (e.g., display device 104). For example,video converter 410 may scale a 640×480 video to the resolution ofdisplay screen 114, which, for example, may have a resolution of1280×720 or 1920×1080. Video converter 410 may perform frame-rateconversion. That is, when a video signal is being converted from onestandard to another (e.g., NTSC to PAL), video converter 410 may adjustthe frame rate in accordance with the specifications of the standards.In situations where the received video signal is already in aprogressive format, video format converter 410 may not need todeinterlace the video signal, and may instead perform other processingsteps, such as scaling the video and/or performing frame-rateconversion. For example, if the original content is determined to beprogressive 320×240 at 30 frames per second, but display screen 114 is1280×720 at 60 frames per second, converter 410 may be scale the videosignal to 1280×720 using vector interpolation and convert it to 60frames per second by repeating every frame once. Alternatively,frame-rate conversion by converter 410 may involve a more advanced formof frame-rate conversion, such as motion-compensated frame-rateconversion. Frame-rate conversion, and more particularlymotion-compensated frame-rate conversion, and its functionalities aredescribed in greater detail in Biswas, et al. U.S. patent applicationSer. No. 11/803,535, which is hereby incorporated by reference herein inits entirety. Video converter 410 may process the video signal in otherways. Additional processing may include any of the techniques discussedbelow in connection with steps 702 and 704 in flow diagrams 700 and 800(FIGS. 7 and 8). These techniques may be chosen based on the detectedresolution of the video signal.

Flow diagram 600 in FIG. 6 shows alternative steps that may be taken byvideo format converter 410 (FIG. 4) to deinterlace and process a videosignal. At step 602, a video signal is received from a video providingdevice. The video signal may be of any suitable format. At step 604, theresolution of the video is determined. Detection or determination of thevideo resolution may occur using any of the techniques discussed inconnection with step 504 of flow diagram 500 (FIG. 5). After determiningthe video resolution, video converter 410 may convert the video signalto its actual resolution at step 606. For the example discussed inconnection with FIG. 3, video converter 410 may revert the transmitted640×480 video signal to a 320×240 video signal. Video converter 410 mayundo any pixel replications performed by the video providing device,thereby recovering the original video signal. Video converter 410 mayignore pixels that it determines are replications of other pixels, orvideo converter 410 may perform another processing technique thatrecovers a video signal with its true resolution.

After obtaining a video signal with its original resolution at step 606,video converter 410 (FIG. 3) may deinterlace and process the convertedvideo signal at step 608. Since the video signal is in its properresolution, a deinterlacer may choose a deinterlacing technique that isbest suited for scaling a video from its original, true resolution(e.g., 320×240 for the image in FIG. 4) to the final resolution of adisplay screen (e.g., display screen 114). Deinterlacing at step 608 mayinvolve any of the types of deinterlacing (e.g., 2D, 3D, etc.) discussedabove in connection with step 506 of flow diagram 500 (FIG. 5).Additional processing may also be applied to the video signal, includingany of the techniques discussed below in connection with steps 702 and704 in flow diagrams 700 and 800 (FIGS. 7 and 8). Any of thesetechniques may be chosen based on the true resolution of the video.

FIGS. 7 and 8 show illustrative flow diagrams 700 and 800 for improvingthe appearance of a low-resolution video on a large-screen display. InFIG. 7, additional processing steps are performed followingdeinterlacing at step 506. Although flow diagram 700 shows deinterlacingusing the steps from flow diagram 500 (FIG. 5), the steps in flowdiagram 600 (FIG. 6) may be used instead (e.g., by replacing steps 502through 506 in FIG. 7 with steps 604 through 608). The particularprocessing techniques that are performed on a video may depend onwhether the video is high-quality (e.g., professionally-generated,low-resolution video on the Internet, videos from portable media player208, etc.) or low-quality (e.g., amateur-generated videos on theInternet, highly compressed videos, etc.). The quality of a video may bedetermined in any suitable way. The quality may be hard-coded orhard-wired if, for example, dock 408 takes input from only one type ofvideo providing device, and the video-providing device typicallyprovides substantially the same quality video. Alternatively, a suitablemetric for assessing the quality of a video may be determined. The videomay be considered high-quality if the calculated metric is higher than acertain threshold.

Low-quality videos may suffer from artifacts such as blocking artifactsand mosquito noise. Blocking artifacts refer to the blocky appearance ofa low resolution video that is typically seen on areas of less detail inthe image. Mosquito noise is a ringing effect, caused by truncatinghigh-frequency luminance and/or chrominance coefficients, typically seenaround sharp edges in the video. These and other artifacts may be causedfrom amateur recording and/or encoding techniques (e.g., using non-idealcompression settings, holding a hand-held camera instead of using atripod, etc.).

For low-quality videos, the compression artifacts may be reduced at step702 (e.g., by converter 410 in FIG. 4). Artifact reduction at step 702may involve reducing one or more types of artifacts (e.g., mosquitonoise, blocking artifacts, etc.). Artifacts may be reduced using one ormore hardware-based or software-based modules. One or more types ofartifacts may be reduced by combining different noise-reducingtechniques into a single module, by cascading various noise-reducingmodules, or using any other suitable technique. In some embodiments,block and mosquito noise reduction may be used at step 702. This can bereferred to as “MPEG noise reduction,” but its application is useful forany compression scheme based on a discrete cosine transform (DCT),including H.264, VC-1, MPEG4, and MPEG2. In some embodiments, 3D videonoise reduction may be used to reduce both temporal and spatial noise.The amount of noise reduction performed at step 702 (e.g., the number ofnoise reducing techniques used, the degree to which each technique isused, etc.) may depend on the assessment of the quality of the video.MPEG noise reduction and 3D video noise reduction are described ingreater detail in Pathak U.S. patent application Ser. No. 11/521,927 andPathak et al. U.S. patent application Ser. No. 11/400,505, respectively,which are hereby incorporated by reference herein in their entirety.Each disclosure also describes a way to assess the quality of the video.The former has a blockiness and mosquito noise measurement, and thelatter has automatic noise estimation. Either, both, or any othersuitable measurement may be used to determine whether a video is high-or low-quality and/or to determine the amount of noise reductionnecessary.

As described above, prior to noise reduction at step 702, the videosignal may include true information about the video content as well asnoise. Because of the poor quality of the video signal, adisproportionate amount of the video information in the signal may benoise information rather than true video information. Thus, after thisnoise is reduced at step 702, a disproportionate amount of the originalinformation may be reduced or even completely removed. Thus, displayingonly the remaining information may not create a pleasant picture, asthere may be very little detail. For example, a low-quality video thatwas taken by a hand-held camera may have a moving picture caused by ashaking camera, even if the background or other parts of the picture aresubstantially unmoving for successive images. Thus, when the video iscompressed, inter-frame compression techniques that would otherwisenotice that the fields/frames are unchanging may not be as effective.For a given data rate or file size, extra bits are used to capture theshaking “noise,” leaving fewer bits for detail and other actualinformation. Noise reduction may reduce the noise caused by the shakingcamera, effectively reducing or removing information from the videosignal. Thus, if the remaining information is displayed to a user, theremay not be enough video information to create a pleasing display. Forexample, the resulting video may have blurred edges, since the originaledges were noisy and removed. Similarly, the resulting video may havelow contrast. In general, once artifacts are removed from noisy areas ofthe picture, there may be very little detail left in those areas.

Accordingly, following noise reduction at step 702, the video signal isenhanced at step 704 (e.g., by converter 410 in FIG. 4). Videoenhancement at step 704 may involve enhancing different aspects of thevideo (e.g., the edges, the color/light contrasts, etc.). Videoenhancement may occur using one or more hardware-based or software-basedmodules. One or more aspects of the video may be enhanced by combiningdifferent video-enhancing techniques into a single module, by cascadingvarious video-enhancing modules, or using any other suitable technique.In some embodiments, video enhancement may involve color remapping. Thatis, certain colors in a video may be mapped into other shades or othercolors. For example, certain shades of green that typically correspondto the color of grass may be remapped to a more vibrant,healthier-looking shade of green. In some embodiments, video enhancementmay involve changing the contrast of colors or light. For example, tomake a picture more vivid, a video processor may increase the lightingcontrast. Color remapping and video contrast enhancement, and theirfunctionalities, are discussed in greater detail in Srinivasan et al.U.S. patent application Ser. No. 11/296,163 and Srinivasan et al. U.S.patent application Ser. No. 11/295,750, respectively, which are herebyincorporated by reference herein in their entirety.

Video enhancement at step 704 may involve adding film grain. Film grainis a high-frequency noise that is naturally present in film, but not indigital video. It is referred to herein as a noise source of anydistribution and magnitude that may be added to a video signal.Typically, film grain is generated by a film grain generator and addedto high-definition digital video. Film grain on high-definition, digitalvideo is used to create a softer, creamier feeling in the picture thatis characteristic of film. This is often accomplished by adding aspatio-temporal noise pattern, a particularly effective way to create“perceptual masking,” which involves the reduction of visual acuity. Forlow-resolution video, on the other hand, the addition of film grain mayestablish the look of texture in a blurred image. Adding film grain maycreate the illusion that there is detail in the picture, even though,due to low resolution and noise reduction at step 702, there actuallymay be very little detail. Film grain may be added to cover up remainingartifacts or other areas of poor visual quality. For example, if a noisepattern with high spatio-temporal frequencies is used, “perceptualmasking” may cause viewers to be less aware or bothered by the remainingartifacts. Film grain generation and addition, and its functionalities,are discussed in greater detail in Balram et al. U.S. patent applicationSer. No. 11/313,577, which is hereby incorporated by reference herein inits entirety.

Videos that are high-quality but low-resolution (e.g.,professional-generated videos from the Internet) may also be enhancedaccording to the techniques described above in connection with step 704.Step 702 may often be skipped, because professionally-generated videostypically do not suffer from substantial compression artifacts. Theabove-described techniques for video enhancement, or any other suitabletechnique, may be used to cover up deficiencies, to simulate detail, tosmoothen blocky areas, to add contrast to the picture, or to provide anyother enhancement that may increase the viewing pleasure of alow-resolution video (e.g., 320×240 video from portable media player208) on a large-screen display (e.g., display screen 114 with 1280×720resolution).

Referring now to FIG. 8, illustrative flow diagram 800 shows analternative embodiment for improving the visual quality of a video. Notethat the steps in flow diagram 800 are the same as those in flow diagram700 (FIG. 7), but arranged in a different order. Although flow diagram800 shows deinterlacing using the steps from flow diagram 500 (FIG. 5),the steps in flow diagram 600 (FIG. 6) may be used instead (e.g., byreplacing steps 502 and 504 with steps 602 through 606 and step 506 withstep 608). Flow diagram 800 shows that deinterlacing does notnecessarily occur before the additional processing steps. In particular,deinterlacing step 506 is shown in flow diagram 800 to occur betweenprocessing steps 702 and 704. However, it should be understood thatdeinterlacing may be performed at any time relative to each noisereduction and video enhancement technique associated with steps 702 and704. That is, flow diagram 800 may be altered such that any of the noisereduction techniques associated with step 702 may be performed at thesame time or after deinterlacing (for low-quality videos), and any ofthe video enhancement techniques associated with step 704 may beperformed at the same time or prior to deinterlacing. Thus,deinterlacing step 506 may also follow processing step 704.

Referring now to FIGS. 9A-9G, various exemplary implementations of thepresent invention are shown.

Referring now to FIG. 9A, the present invention can be implemented in ahard disk drive 900. The present invention may be implemented as part ofthe signal processing and/or control circuits, which are generallyidentified in FIG. 9A at 902. In some implementations, the signalprocessing and/or control circuit 902 and/or other circuits (not shown)in the HDD 900 may process data, perform coding and/or encryption,perform calculations, and/or format data that is output to and/orreceived from a magnetic storage medium 906.

The HDD 900 may communicate with a host device (not shown) such as acomputer, mobile computing devices such as personal digital assistants,cellular phones, media or MP3 players and the like, and/or other devicesvia one or more wired or wireless communication links 908. The HDD 900may be connected to memory 909 such as random access memory (RAM),nonvolatile memory such as flash memory, read only memory (ROM) and/orother suitable electronic data storage.

Referring now to FIG. 9B, the present invention can be implemented in adigital versatile disc (DVD) drive 910. The present invention may beimplemented as part of the signal processing and/or control circuits,which are generally identified in FIG. 9B at 912, and/or mass datastorage 918 of the DVD drive 910. The signal processing and/or controlcircuit 912 and/or other circuits (not shown) in the DVD 910 may processdata, perform coding and/or encryption, perform calculations, and/orformat data that is read from and/or data written to an optical storagemedium 916. In some implementations, the signal processing and/orcontrol circuit 912 and/or other circuits (not shown) in the DVD 910 canalso perform other functions such as encoding and/or decoding and/or anyother signal processing functions associated with a DVD drive.

The DVD drive 910 may communicate with an output device (not shown) suchas a computer, television or other device via one or more wired orwireless communication links 917. The DVD 910 may communicate with massdata storage 918 that stores data in a nonvolatile manner. The mass datastorage 918 may include a hard disk drive (HDD). The HDD may have theconfiguration shown in FIG. 9A. The HDD may be a mini HDD that includesone or more platters having a diameter that is smaller thanapproximately 1.8″. The DVD 910 may be connected to memory 919 such asRAM, ROM, nonvolatile memory such as flash memory and/or other suitableelectronic data storage.

Referring now to FIG. 9C, the present invention can be implemented in ahigh definition television (HDTV) 920. The present invention may beimplemented as part of the signal processing and/or control circuits,which are generally identified in FIG. 9C at 922, a WLAN interface 929and/or mass data storage 927 of the HDTV 920. The HDTV 920 receives HDTVinput signals in either a wired or wireless format and generates HDTVoutput signals for a display 926. In some implementations, signalprocessing circuit and/or control circuit 922 and/or other circuits (notshown) of the HDTV 920 may process data, perform coding and/orencryption, perform calculations, format data and/or perform any othertype of HDTV processing that may be required.

The HDTV 920 may communicate with mass data storage 927 that stores datain a nonvolatile manner such as optical and/or magnetic storage devicesfor example hard disk drives HDD and/or DVDs. At least one HDD may havethe configuration shown in FIG. 9A and/or at least one DVD may have theconfiguration shown in FIG. 9B. The HDD may be a mini HDD that includesone or more platters having a diameter that is smaller thanapproximately 1.8″. The HDTV 920 may be connected to memory 928 such asRAM, ROM, nonvolatile memory such as flash memory and/or other suitableelectronic data storage. The HDTV 920 also may support connections witha WLAN via a WLAN network interface 929.

Referring now to FIG. 9D, the present invention may be implemented in adigital entertainment system 932 of a vehicle 930, which may include aWLAN interface 944 and/or mass data storage 940.

The digital entertainment system 932 may communicate with mass datastorage 940 that stores data in a nonvolatile manner. The mass datastorage 940 may include optical and/or magnetic storage devices such ashard disk drives (HDDs) and/or DVD drives. The HDD may be a mini HDDthat includes one or more platters having a diameter that is smallerthan approximately 1.8″. The digital entertainment system 932 may beconnected to memory 942 such as RAM, ROM, nonvolatile memory such asflash memory and/or other suitable electronic data storage. The digitalentertainment system 932 also may support connections with a WLAN viathe WLAN interface 944. In some implementations, the vehicle 930includes an audio output 934 such as a speaker, a display 936 and/or auser input 938 such as a keypad, touchpad and the like

Referring now to FIG. 9E, the present invention can be implemented in acellular phone 950 that may include a cellular antenna 951. The presentinvention may be implemented as part of the signal processing and/orcontrol circuits, which are generally identified in FIG. 9E at 952, aWLAN interface 968 and/or mass data storage 964 of the cellular phone950. In some implementations, the cellular phone 950 includes amicrophone 956, an audio output 958 such as a speaker and/or audiooutput jack, a display 960 and/or an input device 962 such as a keypad,pointing device, voice actuation and/or other input device. The signalprocessing and/or control circuits 952 and/or other circuits (not shown)in the cellular phone 950 may process data, perform coding and/orencryption, perform calculations, format data and/or perform othercellular phone functions.

The cellular phone 950 may communicate with mass data storage 964 thatstores data in a nonvolatile manner such as optical and/or magneticstorage devices for example hard disk drives HDD and/or DVDs. At leastone HDD may have the configuration shown in FIG. 9A and/or at least oneDVD may have the configuration shown in FIG. 9B. The HDD may be a miniHDD that includes one or more platters having a diameter that is smallerthan approximately 1.8″. The cellular phone 950 may be connected tomemory 966 such as RAM, ROM, nonvolatile memory such as flash memoryand/or other suitable electronic data storage. The cellular phone 950also may support connections with a WLAN via a WLAN network interface968.

Referring now to FIG. 9F, the present invention can be implemented in aset top box 980. The present invention may be implemented as part of thesignal processing and/or control circuits, which are generallyidentified in FIG. 9F at 984, a WLAN interface 996 and/or mass datastorage 990 of the set top box 980. The set top box 980 receives signalsfrom a source such as a broadband source and outputs standard and/orhigh definition audio/video signals suitable for a display 988 such as atelevision and/or monitor and/or other video and/or audio outputdevices. The signal processing and/or control circuits 984 and/or othercircuits (not shown) of the set top box 980 may process data, performcoding and/or encryption, perform calculations, format data and/orperform any other set top box function.

The set top box 980 may communicate with mass data storage 990 thatstores data in a nonvolatile manner. The mass data storage 990 mayinclude optical and/or magnetic storage devices, for example hard diskdrives HDD and/or DVDs. At least one HDD may have the configurationshown in FIG. 9A and/or at least one DVD may have the configurationshown in FIG. 9B. The HDD may be a mini HDD that includes one or moreplatters having a diameter that is smaller than approximately 1.8″. Theset top box 980 may be connected to memory 994 such as RAM, ROM,nonvolatile memory such as flash memory and/or other suitable electronicdata storage. The set top box 980 also may support connections with aWLAN via a WLAN network interface 996.

Referring now to FIG. 9G, the present invention can be implemented in amedia player 1000. The present invention may be implemented as part ofthe signal processing and/or control circuits, which are generallyidentified in FIG. 9G at 1004, a WLAN interface 1016 and/or mass datastorage 1010 of the media player 1000. In some implementations, themedia player 1000 includes a display 1007 and/or a user input 1008 suchas a keypad, touchpad and the like. In some implementations, the mediaplayer 1000 may employ a graphical user interface (GUI) that typicallyemploys menus, drop down menus, icons and/or a point-and-click interfacevia the display 1007 and/or user input 1008. The media player 1000further includes an audio output 1009 such as a speaker and/or audiooutput jack. The signal processing and/or control circuits 1004 and/orother circuits (not shown) of the media player 1000 may process data,perform coding and/or encryption, perform calculations, format dataand/or perform any other media player function.

The media player 1000 may communicate with mass data storage 1010 thatstores data such as compressed audio and/or video content in anonvolatile manner. In some implementations, the compressed audio filesinclude files that are compliant with MP3 format or other suitablecompressed audio and/or video formats. The mass data storage 1010 mayinclude optical and/or magnetic storage devices for example hard diskdrives HDD and/or DVDs. At least one HDD may have the configurationshown in FIG. 9A and/or at least one DVD may have the configurationshown in FIG. 9B. The HDD may be a mini HDD that includes one or moreplatters having a diameter that is smaller than approximately 1.8″. Themedia player 1000 may be connected to memory 1014 such as RAM, ROM,nonvolatile memory such as flash memory and/or other suitable electronicdata storage. The media player 1000 also may support connections with aWLAN via a WLAN network interface 1016. Still other implementations inaddition to those described above are contemplated.

The foregoing describes systems and methods for improving the visualquality of low-resolution video on large-screen displays. The abovedescribed embodiments of the present invention are presented for thepurposes of illustration and not of limitation.

1-147. (canceled)
 148. A method of processing video content for displayon a video display, the method comprising: receiving a video signalhaving a received resolution; detecting an actual resolution of thereceived video signal, wherein the actual resolution differs from thereceived resolution; and processing the received video signal, whereinthe processing comprises selecting one of a plurality of deinterlacingtechniques based on the actual resolution.
 149. The method of claim 148,wherein processing the received video signal comprises deinterlacing thereceived video signal using the selected deinterlacing technique. 150.The method of claim 148 further comprising: reducing artifacts in thereceived video signal; and enhancing the received video signal.
 151. Themethod of claim 148 further comprising: scaling the received videosignal to a second resolution based on the actual resolution, whereinthe second resolution corresponds to a resolution of the video display;and displaying the scaled video signal on the video display.
 152. Themethod of claim 151, wherein scaling the received video signalcomprises: converting the received video signal to a video signal withthe actual resolution; and scaling the converted video signal to a videosignal with the second resolution.
 153. A method of processing videocontent for display on a video display, the method comprising: receivinga video signal having a received resolution; detecting an actualresolution of the received video signal, wherein the actual resolutiondiffers from the received resolution; and processing the received videosignal, wherein the processing comprises selecting one of a plurality ofnoise reduction techniques based on the actual resolution.
 154. Themethod of claim 153, wherein processing the received video signalcomprises reducing artifacts in the received video signal using theselected noise reduction technique.
 155. The method of claim 153,further comprising deinterlacing the received video signal.
 156. Themethod of claim 153 further comprising: scaling the received videosignal to a second resolution based on the actual resolution, whereinthe second resolution corresponds to a resolution of the video display;and displaying the scaled video signal on the video display.
 157. Themethod of claim 156, wherein scaling the received video signalcomprises: converting the received video signal to a video signal withthe actual resolution; and scaling the converted video signal to a videosignal with the second resolution.
 158. A system for processing videocontent for display on a video display, the system comprising: areceiver configured to receive a video signal having a receivedresolution; detection circuitry configured to detect an actualresolution of the received video signal, wherein the actual resolutiondiffers from the received resolution; and processing circuitryconfigured to select one of a plurality of deinterlacing techniquesbased on the actual resolution and to process the received video signal.159. The system of claim 158, wherein the processing circuitry isfurther configured to deinterlace the received video signal using theselected deinterlacing technique.
 160. The system of claim 158, whereinthe processing circuitry is further configured to: reduce artifacts inthe received video signal; and enhance the received video signal. 161.The system of claim 158, wherein the processing circuitry is furtherconfigured to: scale the received video signal to a second resolutionbased on the actual resolution, wherein the second resolutioncorresponds to a resolution of the video display; and display the scaledvideo signal on the video display.
 162. The system of claim 158, whereinthe processing circuitry is further configured to: convert the receivedvideo signal to a video signal with the actual resolution; and scale theconverted video signal to a video signal with a second resolution,wherein the second resolution corresponds to a resolution of the videodisplay.
 163. A system for processing video content for display on avideo display, the system comprising: a receiver configured to receive avideo signal having a received resolution; detection circuitryconfigured to detect an actual resolution of the received video signal,wherein the actual resolution differs from the received resolution; andprocessing circuitry configured to select one of a plurality of noisereduction techniques based on the actual resolution and to process thereceived video signal.
 164. The system of claim 163, wherein theprocessing circuitry is further configured to reduce artifacts in thereceived video signal using the selected noise reduction technique. 165.The system of claim 163, wherein the processing circuitry is furtherconfigured to deinterlace the received video signal.
 166. The system ofclaim 163, wherein the processing circuitry is further configured to:scale the received video signal to a second resolution based on theactual resolution, wherein the second resolution corresponds to aresolution of the video display; and display the scaled video signal onthe video display.
 167. The system of claim 163, wherein the processingcircuitry is further configured to: convert the received video signal toa video signal with the actual resolution; and scale the converted videosignal to a video signal with a second resolution, wherein the secondresolution corresponds to a resolution of the video display.